LLMs/GPT/RL Large Language Models - Generative Pretraining Tranformers - Reinforcement Learning
- 20230712 BP Threads Review: How Meta's New App Stacks Up Against Twitter - Good discussion of the different types of Social Media. Which one is most LLM compatible?
- 20230711 MD Transcript: Ezra Klein Interviews Demis Hassabis
- missing innovations: factuality, robustness, in the realm of planning and reasoning and memory.
- AlphaFold things are scientific tools for experts; at the moment, I think chat bots are more of a fun entertainment thing.
- (chat) could call a bunch of specialized systems and specialized tools.
- 20230713 BP 'Human Beings Are Soon Going to Be Eclipsed' - Brooks comments on Hofstadter, who stikes a decidedly different tone from his article(below) in the Atlantic
- 20230708 BP Goedel, Escher, Bach, and AI - Doug does not mention the fact that chatGPT readily admits: It has no access to the actual book it's asked to comment on.
- 20230706 BP ChatGPT suddenly 'isn't booming anymore,' Google A.I. researcher says—and kids are the big problem
- 20230626 BP In Classrooms, Teachers Put A.I. Tutoring Bots to the Test - Wrong answers; Gives students answers vs. instruction; Significant computing costs
- 20230620 BP YK Yannic Kilcher - 20230620 news - OpenLLaMA on HuggingFace
- 20230605 BP ChatGPT and Generative AI in Search: The Biggest Digital Ad Format Is Ripe for Revolution - Generative Personalized Ads.
- 20230604 BP Financial advisers lean into AI to work better and faster #Investing
- 20230602 BP Why Hollywood Really Fears Generative AI
- 20230527 PL Here's What Happens When Your Lawyer Uses ChatGPT
- 20230525 BP JPMorgan is developing a ChatGPT-like A.I. service that gives investment advice #Investing
- 20230524 BP See why AI like ChatGPT has gotten so good, so fast
- 20230524 BP Can a chatbot help people with eating disorders as well as another human?
- 20230522 BP Bill Gates Says Amazon and Google are Facing a Major Threat
- 20230520 BP 7 Best AI Stocks Of 2023 - MSFT,AMZN,NVDA,AI,GOOGL,MU,TSLA - #Investing
- 20230519 BP Apple bans employees from using ChatGPT. Should you?
- 20230513 BP YK Yannic Kilcher - 20230513 news,
deepfloyd - A MODULAR CASCADED DIFFUSION MODEL,
YOLO Object Detection Model,
Paella,
- 20230513 BP 6 AI stocks to invest in now - MSFT,GOOG,NVDA,AMD,AMZN,AMBA - #Investing
- 20230511 LG Open source AI makes modern PCs relevant, and subscriptions seem shabby -
- 20230430 BP Large language models like AI chatbots seem to be everywhere. If you understand them better, you can use them better., Good Table
- 20230426 BP The Real Problem With Fake Drake
- 20230419 BP Inside the secret list of websites that make AI like ChatGPT sound smart
- 20230417 BP ChatGPT Can Decode Fed Speak, Predict Stock Moves From Headlines pdf - "Bloomberg LP released a large language model for finance last month." yahoo #Investing
- 20230417 BP OpenAI's CEO Says the Age of Giant AI Models Is Already Over
- 20220327 FF GPT-4 Technical Report
- progress on transformers, the type of machine learning model at the heart of GPT-4 and its rivals, lies beyond scaling
2019 GPT2 1.5 B parameters; 2020 GPT3 175 B parameters, 2022 GPT4 ?? parameters
Going Forward: new AI model designs, or architectures, and further tuning based on human feedback
"although OpenAI is keeping GPT-4's size and inner workings secret, it is likely that some of its intelligence already comes from looking beyond just scale. On possibility is that it used a method called reinforcement learning with human feedback, which was used to enhance ChatGPT."
- 20230417 RC ChatGPT Can Give Great Answers. But Only If You Know How to Ask the Right Question - pdf
- 20230416 BP 60 Minutes: Is artificial intelligence advancing too quickly? What AI leaders at Google say
- 20230413 FF Sparks of Artificial General Intelligence: Early experiments with GPT-4 youtube
- 20230406 BP ChatGPT cooks up fake sexual harassment scandal and names real law professor as accused
- 20230403 BP Beyond ChatGPT: Stuart Russell on the Risks and Rewards of A.I.
- 20230400 BP Generative AI at Work - customer support productivity increased 14%; disseminates knowledge; improves customer sentiment, employee retention
- 20230325 BP Godfather of artificial intelligence" talks impact and potential of AI
- 20230323 FF ChatGPT Gets Its "Wolfram Superpowers"!
- BP Kahneman argues that people think "fast" (emotionally) and "slow" (logically/methodically). I suspect that chatGPT is a "fast" path. Whether Wolfram is the "slow" path, I guess we'll see. (Updated) fast AND slow. Look here.
- 20240315 FF Ilya Sutskever
- 20240314 FF GPT-4 Developer Livestream
FF - I do think it already has some grammar. You can check it by setting the first prompt to "You are an expert in financial markets" and then asking it to answer some esoteric questions. At least this is what is implied by this video. The biggest news from this video is explanation of the answer capability
- 20230308 BP Noam Chomsky: The False Promise of ChatGPT
- 20230300 BP Neural Computation - Large Language Models and the Reverse Turing Test
- 20230226 BP Why Do A.I. Chatbots Tell Lies and Act Weird? Look in the Mirror.
- 20221205 BP Stackoverflow: Temporary policy: ChatGPT is banned - Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers.
- Holly
- *20230718 BP Introducing Llama 2, Llama 2: Open Foundation and Fine-Tuned Chat Models
- 20230707 BP OpenLLaMA 13B, 7B, HuggingFace
- *20230706 BP LlamaIndex and the New World of LLM Orchestration Frameworks - combine your own custom data with an LLM — without using fine-tuning or overly long prompts.
- 20230706 BP A New Google AI Research Proposes to Significantly Reduce the Burden on LLMs by Using a New Technique Called Pairwise Ranking Prompting (PRP)
- 20230704 BP - Image Generation 1: Diffusion model - 20230528 GPT-1, GPT-2, GPT-3, InstructGPT / ChatGPT and GPT-4 summary
- 20230608 BP Understanding GPT tokenizers
- 20230601 BP Building a Vector Database to Make Use of Vector Embeddings
- 20230529 BP List of Open Source Large Language Models (LLMs), MPT-7B, Falcon 40B, Huggingface Leaderboard - Google "open source llm"
- 20230527 BP ChatGPT Plus too pricey? 6 websites that let you access GPT-4 for free
- 20230525 BP The False Promise of Imitating Proprietary LLMs
- 20221024 BP Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data, git
- 20230521 BP Forbes, DarkBERT: A Language Model for the Dark Side of the Internet - online cops. This is how china will use it.
- 20230518 BP LIMA: Less Is More for Alignment, decoder - LIMA is based on Meta's 65 billion parameter LLaMA model. The difference is that Meta did only fine-tuning with 1000 selected examples, instead of very extensive training with lots of human feedback (RLHF) like OpenAI.
- 20230517 BP Chat with your CSV: Visualize Your Data with Langchain and Streamlit #Investing
- 20230504 BP Google "We Have No Moat, And Neither Does OpenAI", comments
- 20230423 YK Finetuning Large Language Models
- 20230418 BP Want More Out of Generative AI? Here Are 9 Useful Resources -
Inside My Head,
smartest prompts,
Towards AI,
The AI Podcast,
Visually AI,
GPT for Educators,
Generative AI Courses,
Gradient Ascent,
Learn Prompting,
MIT AI ML Club
- 20230417 BP RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens, huggingface
- 20230330 BP 20230330 - Deep Learning Decade and GPT4 announcement Note: Screenshot 2023-03-30 at 20.04.22.png
- 20180627 BP The Illustrated Transformer
- 20180509 BP Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
- 20230523 YK Tree of Thoughts: Deliberate Problem Solving with Large Language Models - youtube pdf
- 20230611 Forest of Thoughts: Boosting Large Language Models with LangChain and HuggingFace git 4:11
- 20230608 How to write Tree of Thoughts Prompts. - "Decorate" Prompts #Investing
- Imagine three different experts are answering this question.
They will brainstorm the answer step by step reasoning carefully and taking all facts into consideration
All experts will write down 1 step of their thinking,
then share it with the group.
They will each critique their response, and the all the responses of others
They will check their answer based on science and the laws of physics
Then all experts will go on to the next step and write down this step of their thinking.
They will keep going through steps until they reach their conclusion taking into account the thoughts of the other experts
If at any time they realise that there is a flaw in their logic they will backtrack to where that flaw occurred
If any expert realises they're wrong at any point then they acknowledges this and start another train of thought
Each expert will assign a likelihood of their current assertion being correct
Continue until the experts agree on the single most likely location
The question is...
- 1. Carlos is at the swimming pool.
2. He walks to the locker room, carrying a towel.
3. He puts his watch in the towel and carries the towel tightly to a lounger at the poolside.
4. At the lounger he opens and vigorously shakes the towel, then walks to the snack bar.
5. He leaves the towel at the snack bar, then walks to the diving board.
6. Later Carlos realises he has has lost his watch. Where is the single most likely location of the watch?
- 20230604 AI Revolution: Exploring Tree of Thoughts Prompt Engineering -{BP: GPT+ genetic Programming - Portfolio Optimization, Drug Design} #Investing
- 20230521 BP Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling,
Reprompting: Chain-of-Thought (CoT) Recipes without Human Intervention
- 20230517 BP GPT-4 Prompts to Filter Top Stocks Using AI- Hedge Fund Manager #Finance
...
- He fed GPT-4 data points for the 30 Dow stocks in the 12 months through May 3, and prompted it to pick the top five stocks it expects will perform well in the next 12 months.
It picked three of the same stocks Patel already held: Microsoft, Visa, and Apple. All three stocks were up by about 22%, 19%, and 22% respectively through May 10, and among the index's top-seven performers. It suggested two additional stocks that he had not picked: Merck and Procter and Gamble.
After reviewing his prior analyses of those two stocks, he realized that he had concluded they were also strong buys but had left them out.
- he needed to prompt it through a series of filters. And when you do this, he said, be elitist, which means filter it through the top and best sources.
Below is a series of prompts in the order Patel ran through after copying and pasting the Dow stocks.
- Imagine you are an outstanding fund manager such as Warren Buffett and Nobel Prize Winner Eugene Fama. Tell me the best five stocks to hold for the next 12 months based on those fund managers and the data about the companies provided here. You will have to weigh the importance of the data and consider all the factors. Provide the stock names and your reasoning. You will need to know the Alpesh Value/Growth score is a measure of valuation and company growth and the higher the number, the better. But it is for you to decide the importance of each individual datapoint.
- Would your answer be different if I told you we are in a macroeconomic environment where interest rates are rising? If so, what would be the top 5 stocks now and why? (At this point, GPT-4 replaced one of the previous five with a new company.)
- Now, Imagine you are an outstanding fund manager such as Warren Buffett and Nobel Prize Winner Eugene Fama, and you are well versed in the top most influential and important academic literature on stock price forecasting. Tell me the best five stocks to hold for the next 12 months based on those fund managers, that academic literature and the data about the companies provided here. And remember interest rates are rising. You will have to weigh the importance of the data and consider all the factors. Provide the stock names and your reasoning, and remember I told you interest rates are rising.
- Which factors in the data I provided you, do you consider most important and why?
- What data do you think you need that I did not provide which would have helped you to make better forecasts more accurately given you are to be Warren Buffett, Eugene Fama and well versed in the most important academic literature on stock forecasting.
- Detail the most important literature on stock forecasting which you considered in your answers to my questions when I asked you to consider it.
- A key thing to remember when experimenting with this tool is that you don't want ChatGPT to give the data points an equal weighting. Otherwise, it could end up being a mathematical exercise, where it just adds up the columns, Patel said. Instead, you need to ask it to weigh the importance of each data point using the prompted filters. All decision-making, human or artificial intelligence, is based on input and weighing that input, and it's the latter that is the intelligence part, he added.
Another cautionary tale is not to assume it's including every bit of previous data you gave it. Instead, ask it to reference or consider the previous data, he noted.
- 20230326 11 Tips to Take Your ChatGPT Prompts to the Next Level
Apple News
- Learn Prompting Competition May 26th
- Language Model Query Language
- 20230414 FF FUTUREPEDIA: THE LARGEST AI TOOLS DIRECTORY
- 20230607-8 Snorkel Conference
2022
- 20230513 YK H20 - llmstudio git, h2ogpt git, h2o open assistant
- 20230423 YK The Embedding Archives: Millions of Wikipedia Article Embeddings in Many Languages
- Arize 20230425 BP observe reception
- 9:30 AM-10:00 AM video mpowering the Engineer: a Conversation with OpenAI About chatGPT and its Implications - Boris Power, Aparna Dhinakaran
- 10:00 AM-10:30 AM Prompt Engineering in the Real World - Jared Zoneraich also this
- 10:30 AM-11:00 ppt AM How LlamaIndex Brings your Data to LLM's - Jerry Liu #Investing SEC 10-K
- 11:30 AM-12:00 PM video Hugging Face: A Practical Perspective on using LLMs - Rajiv Shah
- 9:00 AM-9:30 AM video Evaluating Models: Traditional ML and LLMs - a Conversation with Anthony Goldbloom (Kaggle) #Gamifying
...
- Kaggle - Random Forest to Transformer based methods.
- Blind comparisons vs paper submission
- 1) Hosted notebooks in order to share code.
- 2) ML Competitions
- 3) Public Dataset.
- Use holdout data to prevent overfitting.
- Live Leaderboard for real time.
- Throw tests cases and then rescore.
- Can only pick two submissions on last test.
- move from 9th->100th place
- This affects papers.
- Most real world algorithms are overfit!!!
- Random Forest became dominant****
- 2012 - Gradient Boosting (XGBoost is dominant on small structured datasets)
- 2012 - Annus Mirabilus - Fei Fei Li's Stanford competition - Hinton's team won.
- Computer Vision classification
- Object Detection - count coke cans
- Segmentation
- Radiology, Autonomous Vehicles unlocked by this
- 2015 - Keras Framework made it possible for others to do
- 2017 - Diffusion Models -
- 2018 - Transformer - Attn is all you need, BERT
- GPT-3
- chatGPT
- Computer Scientists have won (vs. statisticians)
- Transformers do well - proteomics
- Smaller structured problems - Gradient boosting machines.
- Transfer learning does not work on structured data.
- No structured data breakthroughs.
- Now: Pull structured data from unstructured data
- RegEx -> GPT-4
- Entity recognition - Jeff Bezos stepped down.
- F1 0.35 - GPT3 vs .85 for Hugging Face. (Longformer?)
- May use GPT-4 for labeling
- Longformer - train to predict next word.
- LLMs do summarization well - Abstractive Summarization (vs. subtractive)
- Summarization or Hallucination?
- Use one LLM to evaluate another LLM response.
- Structured
- Extract latent structure.
- DOLLY is example.
- Use GPT3/4 or train own internal model?
- Ans: own model: GPT4 is too slow and expensive.
- Use GPT-4 for Labeling.
- Distillation - Keep asking same question (names of people)
- Send LLM 50 examples of how to take names out.
- Then just pull out the active parts of the Model used to answer question.
- Google has strategy Tax and Bureaucratic overhead.
- 9:30 AM-10:00 AM video Intro to Relative Representations - Luca Moschella
- 10:00 AM-10:30 video AM Behind the Scenes of Phoenix: Unraveling Embeddings using UMAP and HDBSCAN
#Investing
- ppt Three Trends in AI Research - Brian Burns
- 1:30 PM-2:00 PM video An Autonomous Agent is Born - A Conversation with the Founder of BabyAGI
...
- 9:00 AM Keynote: The New Wave of ML - How Generative AI Has Forever Changed ML
- 10:30 AM-11:00 AM Optimizing through Automation: End-to-End Data Quality Assurance and Model Deployment Nina Lopatina, Yuanbo Wang
- 12:00 PM-12:30 PM ML Observability Made Easy: A Beginner's Guide to Monitoring Your Models Amber Roberts
- 2:00 PM-2:30 PM Training Billion Parameter LLMs Hanlin Tang
- 2:30 PM-3:15 PM Generative AI in Bioengineering Use Cases Haseeb Khan, Markus Buehler, Archit Khosla
- =========================
- 10:30 AM-11:00 AM Building a Generative AI Company - Jasper's Journey
- 11:00 AM-11:30 AM Future of Agents with Harrison Chase, Founder of LangChain
- 11:30 AM-12:00 PM The Most Overlooked Problem in NLP Development: Common Pitfalls in Data Labeling and How to Fix Them
- 12:00 PM-12:45 PM Generative AI Startups: New Opportunities, Unique Challenges, and Big Visions
- 12:30 PM-1:00 PM A Framework for Data Storytelling
- 1:00 PM-1:30 PM Label Data with Premiere Quality and Productivity Using Datasaur
- 1:30 PM-2:15 PM Building Real Time Inference Pipelines with Ray Serve
- Databricks
- Gits - gpt4free, gpt-steroid, azure-search-openai-demo
- lamini - lamini
- Langchain - py, tutorial Lucidate
- Lucidate - Transformers & NLP, neural networks
- Chain of Thought, React, Tree of Thoughts, Forest of Thoughts
- NFX - ai-startup-litmus-test - Generative-Tech-Stack
- OpenAI - papers,
developer,
chatgpt,
cookbook,
Third-Party Plugins,
- SingleStore
- trulens - trulens
- 20230601 ChatGPT - in its own words
- Programming ChatGPT - in its own words